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1.  GENERAL  COMMENTS  AND  SUMMARY  OF  ACCOMPLISHMENTS 


It  has. often  been  Voted  In  the  literature  of  applied  probability 


(for  example,  see  Gross  and  Harris,  1985)  that  a  lack  of  appropriate 


solution  procedures  frequently  prevents  the  application  of  major 


theoretical  advances.  ^In  the  study  of  complex  large-scale  systems. 


analysis  aimed  at  obtaining  exact  answers  becomes  especially 


ineffective.  Typically,  the  opportunity  to  study  real  systems 


depends  upon  the  ready  access  to  either  closed-form  answers  or 


computer-based  numerical  routines.  There  has  indeed  been  some 


important  progress  made  in  imp  leaientat  ion  but  they  are  only  the 


initial  steps 


Various  numerical  procedures  and  approximation  techniques  have. 


in— fa et-,.^  been  used  over  the  years  for  deriving  information  on  the 


major  performance  measures  of  stochastic  systems.  But,  at  this 


stage,  the  major  point  is  that  it  isnow  realized  that  the  ready 


availability  of  inexpensive  and  friendly  computing  is  making  it  a 


lot  easier  to  bring  the  theory  much  more  into  the  mainstream  of 


real-problem  solution  methods. 


We  have  had  a  particular  interest  in  providing  an  assessment  of 


the  impact  that  recent  advances  in  numerical  procedures  and  their 


computer  implementation  may  have  on  the  effective  application  of 


probability  methods  in  the  modeling  of  dynamic,  large-scale  systems. 
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stochastic  models  with  special  attention  to  algorithmic  feasibilty 
over  a  realistic  range  of  parameter  values.  Professor  Neuts  is 
able  to  avoid  transform  solutions  and  complex  variable  methods  for 
prime  classes  of  problems  by  the  creative  use  of  iterative 
(computer-based)  matrix  methods.  But,  it  should  be  clear  that  such 
methods  are  still  very  heavy  on  computations.  The  obstacles  to 
reasonable  solution  using  Neuts-type  arguments  are  often  formidable, 
though  the  effective  use  of  contemporary  computer  technology  has 
made  them  quite  attractive  for  models  of  dynamic  systems  under 
uncertainty.  However,  the  very  nature  of  the  underlying 
complexity  of  such  models  will  often  diminish  their  usefulness, 
particularly  since  it  may  be  impossible  to  estimate  population 
parameters  in  any  practical  way.  A  major  direction  of  our 
research  has  been  to  find  related  computationally  feasible  methods, 
for  which  statistical  inference  procedures  are  easier  to  develop. 

The  numerical  analysis  of  queueing  systems  carried  out  by  Neuts, 
using  phase-type  distributions,  can  be  identified  as  an  important 
approximating  technique.  The  incorporation  of  these  distributions 
can  be  viewed  as  a  system  of  approximation,  in  that  they  may  be 
used  for  approximating  interarrival  and  service  times.  Our 
generalized  hyperexponential  (GH)  distributions  are  an  important 
relative  of  the  phase  types  (see  Harris  and  Sykes,  1985),  and 
their  use  in  single-server  queueing  systems  is  well  documented  in 
Harris  (  1985  )  . 
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The  distributions  are  linear  combinations  of  exponential  CDFs 

with  mixing  parameters  (positive  and  negative)  that  sum  to  unity. 

The  denseness  of  the  class  GH  with  respect  to  the  class  of  all 

CDFs  defined  on  (0,00)  has  been  established  in  Botta  and  Harris 

(1985)  by  showing  that  a  GH  distribution  can  be  found  that  is  as 

close  as  desired,  with  respect  to  a  suitably  defined  metric, 

to  a  given  CDF.  The  metric  induces  the  usual  topology  of  weak 

convergence  so  that,  equivalently,  there  exists  a  sequence  (G  ) 

n 

of  GH  CDFs  that  converges  weakly  to  any  CDF.  The  result  follows 
from  a  similar  well-known  result  for  weak  convergence  of  Erlang 
mixtures.  Various  set  inclusion  relations  have  also  been  obtained 
relating  the  GH  distributions  to  other  commonly  used  clases  of 
approximating  distributions  including  generalized  Erlang,  mixed 
generalized  Erlang,  those  with  reciprocal  polynomial  Laplace 
transforms,  those  with  rational  Laplace  transforms,  and  phase-type 
dlstribut  ions . 

The  nonlinear  optimization  routine  we  have  developed  for  the 
likelihood  estimation  of  the  GH  parameters  is  described  in  Harris 
and  Sykes  (1985).  Earlier  papers  by  Kaylan  and  Harris  (1981)  and 
Mandelbaum  and  Harris  (1982)  provide  an  interesting  history  of  the 
evolution  of  the  algorithm  and  its  original  application  to  mixtures 
of  exponential  and  Weibull  densities. 

Validation  is  also  an  Integral  part  of  approximation  procedues. 


if 
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IC  is  needed  to  support  the  applicability  of  a  technique  and  the 
reliability  of  results.  An  applied  scientist  has  constantly  to 
evaluate  the  trade-off  between  the  ease  of  application  of  a 
particular  technique  and  the  accuracy  of  the  ensuing  results. 
Experimentation  is  the  most  common  validation  technique  for 
approximations,  and  a  very  common  form  of  experimentation  is 
digital  simulation.  Most  authors  have  estimated  approximation 
accuracy  by  the  relative  size  of  the  percentage  difference 
between  simulated  and  approximate  results.  Very  few  of  them  have 
resorted  to  a  statistical  analysis  of  simulated  results  and 
provide  information  such  as  confidence  bounds  on  their  estimates. 
Clearly  wider  use  of  key  statistical  techniques,  for  point  and 
interval  estimation,  should  be  made  when  simulation  is  preferred. 
We  note  then  that  some  of  our  recent  research  has  been  directed 
at  the  development  of  optimal  simulation  sampling  designs  for  such 
computer-based  experiments  (Harris,  1985b). 

Probability  researchers  have  been  widely  critized  for  studying 
systems  that  are  not  relevant  to  the  real  world.  However,  it 
seems  that  this  criticism  is  largely  due  to  the  complexity  of 
available  results  in  the  literature  rather  than  due  to  the  system 
themselves.  If  one  looks  at  some  of  the  applied  areas  of  queueing 
one  finds  more  complex  systems  than  those  in  the  general  research 
and  applied  probability  literature.  The  distinction  is  in  the 
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nature  of  analysis.  The  results  found  in  the  applied  area 
literature  are  applicable,  though  approximate.  A  large  percentage 
of  results  found  in  the  general  literature  is  less  useful,  though 
rigorous.  Therefore  if  we  want  the  major  methods  of  applied 
probability  to  be  important  problem-solving  tools,  then  approxi¬ 
mation  techniques  should  be  put  to  increasing  use  whenever  necessary 
and  possible.  The  trend  over  the  past  decade  or  so  has  been  in 
this  direction,  and  there  is  every  reason  to  believe  that  this 
pattern  is  going  to  continue  further  in  the  coming  years,  bringing 
more  reliability  and  applicability  for  the  techniques  used. 

Additional  project  support  was  provided  by  visiting  Research 
Fellow  Martin  Krakowski.  His  work  emphasized  some  specific 
aspects  of  modeline  complex  problems  in  optimal  system  design 
and  reliability  under  uncertainty. 

Krakowski's  work  on  the  relevation  and  related  transforms 
fits  very  naturally  into  the  general  theme  of  the  project.  The 
relevation  transform  generates  an  entropy-like  expression  which 
appears  to  be  useful  in  predicting  changes  in  longevity  for  systems 
affected  by  aging  ir  1  i  f e - i mp r o v i ng  treatments.  His  work  had  a 
special  focus  on  approximations  and  numerical  problems,  to  match 
up  with  the  overall  problem  and  application  thrust  of  the  rest 
of  the  project. 
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As  noted,  the  use  of  linear  (not  necessarily  convex)  combinations 
of  exponential  distributions  as  approximants  permits  the  relatively 
easy  solution  of  many  problems.  We  have  since  learned  that  such 
densities  are  in  fact  dense  in  the  class  of  all  probability 
distributions  and  that  some  of  our  previous  work  on  parameter 
estimation  for  Weibull  mixtures  can  be  adjusted  for  the  nonconvex 
combination  of  negative  exponential  distribution.  Two  papers 
were  prepared  on  the  problem  in  project  year  1.  The  first  one 
documented  a  Jacobi-like  nonlinear  optimization  algorithm  for 
the  max imum- 1  ike  1 ihood  estimation  of  parameters  of  generalized 
mixed  exponential  distributions.  The  second  manuscript  provided 
an  important  application  of  this  comprehensive  family  of 
probability  functions  to  a  specific  class  of  problems  frequently 
encountered  in  maintenance,  queueing  and  network  systems. 

The  MLE  algorithm  documented  in  Harris  and  Sykes  obtains  one 
answer  from  among  a  potentially  large  set  of  local  solutions. 

This  work  details  the  development  and  testing  of  a  practical 
procedure  for  the  MLE  of  the  parameters  of  generalized  exponential 
mixtures.  The  algorithm  offered  for  the  problem  guarantees  that 
the  result  will  provide  a  legitimate  probability  function  of 
the  prescribed  type.  Extensive  testing  has  been  performed  and 
results  are  very  favorable:  convergence  is  rapid  and  the  use  of 
computer  resources  rather  limited. 


A  revision  version  of  the  original  project  technical  report 
number  UVA/525393/SE84/ 102  has  been  accepted  for  publication  in 
the  Nava^  Quarterly  and  should  be  appearing 

around  the  middle  of  1986. 

In  the  second  of  these  papers  (Harris,  1985),  an  effective  way 
was  presented  for  the  estimation  of  the  stationery  waiting  time 
distribution  of  the  general  single-server  queueing  model  GI/G/1. 
The  approach  relies  on  obtaining  a  generalized  exponential  mixture 
as  an  approximation  for  the  distribution  of  the  service  times. 

This  is  done  using  the  Harris/Sykes  nonlinear  optimization 
algorithm  as  discussed  above.  The  method  is  very  well  suited 
for  obtaining  the  delay  distribution  beginning  from  raw 


The  metric  induces  the  usual  topology  of  weak  convergence  so  that, 

equivalently,  there  exists  a  sequence  (G  )  of  GH  CDFs  that  converges 

n 

weakly  to  any  CDF.  The  result  follows  from  a  similar  well-known 

result  for  weak  convergence  of  Erlang  mixtures.  Various  set 

inclusion  relations  were  also  obtained  relating  the  GH 

distributions  to  other  commonly  used  classes  of  approximating 

distributions  including  generalized  Erlang  (GE),  mixed 

generalized  Erland  (MGE),  those  with  reciprocal  polynomial 

Laplace  transforms  (K  ),  those  with  rational  Laplace  transforms 

n 

(R^),  and  phase— type  (PH)  distributions.  A  brief  survey  of 
the  history  and  use  of  approximating  distributions  in  queueing 
theory  was  also  included. 

In  the  second  paper,  by  Harris,  there  is  a  concern  for  the 
construction  of  optimum  model-sampling  simulation  experimental 
designs.  The  accurate  estimation  of  the  distribution  of  any 
random  variable  by  computer  is  almost  always  difficult, 
especially  when  reasonable  accuracy  is  desired  for  its  tail 
probabilities.  If,  in  addition,  each  element  of  the  computer 
generated  pseudo-random  sequence  is  itself  the  result  of  a 
stochastic  limit  or  possibly  a  functional  of  a  cont inuous-t ime 
process,  it  becomes  most  complicated  to  assess  the  final 
statistics. 


m 


In  this  paper,  we  focused  on  the  estimation  of  tail  probabilities 
for  the  distribution  function  of  the  maximum  on  the  unit  interval 


1.2.1  The  following  manuscripts  were  completed: 

Harris,  C.M.,  and  Sykes,  £.,  "Maximum  Likelihood  Estimation  for 
Generalized  Mixed  Exponential  Distributions,"  Report  No. 
UVA/525393/SE/84/ 102 ,  accepted  for  publication  in  Naval_ 

*  expected  to  appear  mid- 1986. 

Harris,  C.M.,  "A  Note  on  Mixed  Expoential  Approximations  for 

GI/G/1  Waiting  Times,"  Report  No.  UVA/525393SE/84/ 101 , 
published  in  Com£uters  and  Operations  Research,  Volume  12, 

No.  3,  pp.  285-289,  1985. 

Botta,  R.F.  and  Harris,  C.M.,  "Characterizations  of  Generalized 

Hyperexponential  Distributions,"  Report  No.  UVA/ 5 2 53 93 / SE85 | 107 
accepted  for  publication  in  Queueing  §2£££!B1  • 

22:^  22Ei:i£2£l£2£ »  expected  to  appear  mid  -  1986. 

Harris,  C.M.,  "Comments  on  Estimating  Quantiles  for  Gaussian 
Functionals  by  Simulation,"  Technical  Report  No. 

UVA I  525393  I SE85 I  109  ,  currently  pending  review  with 
Journa^_o  f  _S  t a  t^s  t^£a^_Com£u ta  t  ion_and_S^mu l^a  t^on  . 


1.2.2  The  following  talks  were  given: 

Harris,  C.M.,  "Likelihood  Estimation  of  the  Parameters  of 

Generalized  Exponential  Mixtures,"  Fall  1983  Meeting  of 
ORSA/TIMS,  Orlando,  Florida,  7  November  1983. 

Harris,  C.M.,  "Likelihood  Estimation  of  the  Parameters  of  Generalized 
Exponential  Mixtures,"  Department  of  Computer  Systems  and 
Production  Management,  University  of  Toledo,  26  January  1984. 

Harris,  C.M.,  "Mixtures  of  Discrete  Distributions,"  Department 

of  Computer  Systems  and  Production  Management,  University 
of  Toledo,  27  January  1984. 

Harris,  C.M.,  "Mixed  Exponential  Approximations  for  Gl/G/1 

Waiting  Times,"  Operations  Research  and  Systems  Analysis 
Curriculum,  University  of  North  Carolina  at  Chapel  Hill, 

5  April  1984. 

Harris,  C.M.,  "Mixed  Exponential  Approximations  for  Gl/G/1 

Waiting  Times,"  Operations  Research  Seminar,  University 
of  Delaware,  27  April  1984. 

Harris,  C.M.,  "The  Modeling  and  Application  of  Mixtures  of 

Exponential  Distributions,"  to  Department  of  Mathematical 
Sciences,  The  College  of  William  and  Mary,  Williamsburg, 
Virginia,  26  October  1984. 


Harris,  C.M.,  "The  Modeling  and  Application  of  Mixtures  of 

Exponential  Distributions,"  to  Department  of  Mathematical 
Sciences,  The  Johns  Hopkins  University,  Baltimore,  Maryland, 
7  December  1984. 

Harris,  C.M.,  "The  Modeling  and  Application  of  Mixtures  of 

Exponential  Distributions,"  to  jointly  sponsored  seminar 
of  the  Departments  of  Industrial  Engineering  and 
Mathematical  Sciences,  Clemson  University,  Clemson,  South 
Carolina,  20  February  1985. 

Harris,  C.M.,  and  Sykes,  E.A.,  "Likelihood  Estimation  for 

Generalized  Mixed  Exponential  Distributions,"  Spring  1985 
Meeting  of  ORSA/TIMS,  Boston,  Massachusetts,  invited 
session  sponsored  by  ORSA/TIMS  Applied  Probability  Group, 

29  April  1985. 
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1.3: 

As  a  result  of  our  early  research  in  year  1,  we  decided  to 
move  up  some  work  proposed  for  Task  C.  Of  particular  interest, 
there  was  the  unit  on  the  solution  of  multi-dimensional  difference 
equations.  In  the  proposal  we  noted  that  this  problem  was  closely 
related  to  some  of  those  in  Task  A  in  the  sense  that  the  usual 
solution  to  a  stochastic  model  with  a  multivariate  state  space 
was  presented  as  a  transform.  So  we  have  worked  on  developing 
a  strategy  to  convert  multi-dimensional  problems  down  to  much 
simpler  ones. 

The  primary  application  for  us  of  multi-dimensional  difference 
equations  has  been  the  steady-state  solution  of  networks  of 


Markovian  queues.  A  very  Important  problem  that  has  been  of 
special  concern  is  the  solution  of  queues  with  feedback  and  the 
estimation  of  their  retrial  parameter. 

1.3.1  The  following  manuscript  was  completed: 

Harris,  C.M.,  and  Hoffman,  K.L.,  "Estimation  of  a  Caller  Retrial 
Rate  in  Communication  Networks  with  Feedback,"  accepted 
for  publication  in  the  Euro£ean  J£U£nal^  ££ 

Research,  to  appear  in  August  1986. 

1.3.2  The  following  talk  was  given: 

Harris,  C.M.,  on  "Queueing  Analysis  of  Communication  Networks 
with  Overflow  and  Retrial,"  George  Mason  University 
Engineering  Seminar,  28  November  1983. 
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Throughout  the  contracted  period  Martin  Krakowski's  work 
has  been  concrned  with  the  Joint  use  of  ergodic  variables  and 
of  omni-transforms  (both  terms  are  ad  hoc  designations)  to  the 
problems  of  stationary  random  processes.  The  essence  of  the 
conservation  method  as  applied  to  stochastic  models  consi'sts  of 
the  selection  of  one  or  more  random  variables  each  of  which  has 
an  expected  change  of  zero  over  any  arbitrary  interval  (which 
can  be  taken  to  be  finite,  very  long  or  infinitesimal);  we 
refer  to  such  a  variable  as  an  ergodic  variable.  The  omni¬ 
transform  of  a  random  variable  Y  (in  particular  Y  can  be  an 
ergodic  variable)  is  the  expectation  of  an  arbitrary  (though 
well  enough  behaved  to  permit  differentiation  or  any  other 
needed  formal  operations)  function  of  Y.  Simple  as  the 
notion  of  the  omni-transform  is,  its  consequence  and  wide 
reach  of  applications  are  surprising;  complex  theorems  are  at 
times  replaced  by  short  and  incisive  arguments.  The  omni¬ 
transform  is  applicable  to  models  other  than  queueing,  e.g.  to 
some  models  treated  by  Feller  in  his  Introduction  to  Probability 
Xbjep_ry ,  Volume  1.  The  two  notions,  ergodic  variables  and  omni¬ 
transform,  go  well  together  because  if  Y  is  an  ergodic  variable, 
then  so  is  an  arbitrary  function  of  Y.  The  conservation  approach 
to  single-server  queues,  in  particular  to  the  M/G/1  queueing 
model,  combines  these  two  notions.  So  does  the  treatment  of  the 
renewal  model,  which  has  a  bearing  on  some  aspects  of  the  g/g/1 
model. 
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1.4.1  The  following  manuscripts  were  completed: 

Krakowski,  Martin,  "Omni  Transforms:  Applications  to  Renewal 

Theory,"  Report  No.  UVA/5253/SE84/ 103 . 

Krakowski,  M.,  "The  Omni-Transform  in  Single-Channel  Queues," 
Technical  Report  No.  UVA/525393/SE85/ 106 ,  accepted  for 

publication  in  the  Annal^s  of  Ogerat^ons  Research, 

expected  to  appear  in  late  1986. 


1.4.2  The  following  talks  were  given: 

Krakowskl,  Martin,  On  "Quick  and  Clean  Methods  in  Queueing 

Theory,"  Department  of  Systems  Engineering,  University 
of  Virginia,  17  February  1984. 

Krakowskl,  Martin,  "The  Omni— Trasf orm  in  Single-Channel  Queues 
invited  talk  at  ORSA/TIMS  Conference  on  Statistical  and 
Computational  Problems  in  Probability  Modeling, 
Williamsburg,  VA  9  January  1985. 
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